1. Field of the Invention
This invention is directed to a system for capturing and retrieving audio data and corresponding hand-written notes. In particular, the invention is directed to an electronic notebook that improves the way users record, review, retrieve and reuse notes and related audio data.
2. Description of Related Art
Note-taking is practiced by most information workers, including scientists, executives and students. Reasons for note-taking vary. Students take notes in the classroom and review them later to prepare for exams. Scientists write notes in lab books to record experimental results. Executives take notes in meetings to record action items. Most workers consider notes to have some historical value.
Some workers use lap-tops or Personal Digital Assistants (PDAs) for note-taking, but the majority of people use paper notebooks. Paper notebooks provide greater freedom of expression due to their free-form input as opposed to a structured keyboard input. On the other hand, paper notebooks are hard to organize by topic, particularly the bound or spiral types that are commonly used. It is also difficult to find specific information in bound notebooks, since only a manual, linear search is possible. Finally, while paper notebooks are easy to carry and use, they are not integrated with digital data.
Several systems have been designed for real-time note-taking or annotation of previously recorded audio or video.
Marquee by Karon Weber and Alex Poon is a tool for annotating video with hand-written notes and keywords. Notes are synchronized to the video by creating time zones using a horizontal line gesture. Keywords are defined by circling words in notes, and are applied to regions by a corresponding keyword region. Keywords are manually converted to text for subsequent indexing.
Filochat by Steve Whittaker, Patrick Hyland and Myrtle Wiley is a note-taking system in which audio is recorded and indexed by hand-written notes. A header for each new section of notes is automatically generated. Each header contains the time and date as well as a region in which to write topics and names. Audio is accessed by selecting a seek option and gesturing at notes.
Classroom 2000 by Gregory Abowd, Chris Atkeson, Ami Feinstein, Yusuf Goolamabbas, Cindy Hmelo, Scott Register, Nitin Sawhney and Mikiya Tani is a system for combining a classroom lecture, previously made visual materials for the lecture, and hand-written notes. The audio for the lecture is recorded and indexed by viewgraph. Hand-written notes are made on a digital overlay of the viewgraph. Retrieval of notes is done by searching text in the viewgraphs.
The Paper-Based Audio Notebook by Lisa Stifelman allows a user to capture and access an audio recording using notes written on paper. The audio recording is synchronized to each page of hand-written notes, and audio playback can be initiated by either pointing at notes or by using an audio scrollbar.
EPO Publication 0 495 612 A2 entitled "A Data Access System", Michael G. Lamming, published Jul. 22, 1992 describes a note-taking system based on a notepad computer with an integrated audio/video recorder. As the user types on the keyboard or writes with the stylus or similar input instrument on the notepad computer, each character or stroke that is input by the user is invisibly time-stamped by the computer. This activity results in the creation of meeting "notes". The audio/video stream is also continuously time-stamped during recording. When playback is desired, the meeting notes as finally created are presented to the user. To play a section of recording back, the user selects part of the note (perhaps by circling it with a stylus) and invokes a "playback selection" command. The computer then examines the time-stamp and "winds" the record to the corresponding place in the audio/video recording, where it starts playing--so that the user hears and/or sees what was being recorded at the instant the selected text or strokes were input. With a graphical user interface, the user may input key "topic" words and subsequently place check marks by the appropriate word as the conversation topic veers into that neighborhood.
Tivoli by Elin Pederson, Kim McCall, Thomas Moran, and Frank Halasz simulates whiteboard functionality on a Liveboard. Editing functions such as insertion and deletion are controlled by gestures.
The Meeting Capture System records audio synchronously with notes taken on a Liveboard. Audio playback is initiated by gesturing at Liveboard notes, or using an audio timeline.
Audio segmentation by Lynn Wilcox, Don Kimber and Francine Chen defines a means for separating speakers and sound in real-time or in recorded audio. The segments corresponding to different speakers are displayed on an audio timeline using different color bands or separate timelines for each speaker.
Listener by Debby Hindus, Chris Schmandt and Chris Horner is a system which displays the structure of a phone conversation and allows the user to save selected speaker comments.
These systems have proposed various approaches to solving the need for properly organized and searchable handwritten notes and related audio data. However, none have met the need effectively.